Skip to content

Compress native prebuilds with verified decoding - #1015

Closed
John-David Dalton (jdalton) wants to merge 1 commit into
microsoft:mainfrom
jdalton:feat/compressed-prebuilds
Closed

John-David Dalton (jdalton) wants to merge 1 commit into
microsoft:mainfrom
jdalton:feat/compressed-prebuilds

Conversation

@jdalton

@jdalton John-David Dalton (jdalton) commented Nov 10, 2026 •

Copy link
Copy Markdown

Compress .node addons to reduce the package overhead in #864. The verified rerun shows 2.51% less unpacked data and 0.78% less download data, with slower loading. Windows DLLs, executables, and debug symbols make up 95.1% of unpacked bytes and remain unchanged.

This proposal keeps the existing single-package distribution. The alternative is #1016, which installs only the current platform's files; both approaches address #864 with different savings and tradeoffs.

Complete package and download savings compared with the addon-only subset

The 61.80% addon reduction applies to only 4.16% of the package. The download is already gzip-compressed. Each chart pair uses its own zero-based scale.

Given the 0.056 MB download saving and added loading/cache work, the current compression implementation offers a limited tradeoff. Platform-specific distribution in #1016 is the more promising direction for reducing installed size.

Verified sizes and loading cost
Measure Published Proposed
Complete unpacked package 26,907,504 B 26,232,181 B
npm download 7,152,900 B 7,097,383 B
Eight .node addons only 1,118,952 B 427,483 B
Fresh-process load, existing cache 2.56 ms 5.67 ms
Fresh-process load, new cache 2.56 ms 46.55 ms

The macOS ARM64 decoded cache adds 137,384 B, reducing the net file-byte saving to 2.00% after first use. These are logical file sizes, excluding dependencies and package-manager stores. No startup speedup is claimed.

The rerun used a fresh node-pty@1.2.0-beta.18 download verified against npm's SHA-512 integrity. Repacking the baseline reproduced that archive exactly. Unrelated packed files retain identical sizes and hashes. Size results match the previous run exactly.

Timings are medians of 21 fresh processes per mode on macOS ARM64 / Node 26.12.0. Every process also spawned a real PTY. Node startup and PTY spawning are excluded from load timings; OS caches were not flushed. Linux and Windows startup remain unmeasured.

Raw measurements and file inventories.

Implementation and validation
  • Brotli on macOS; Zstd on Linux and Windows, using built-in Node codecs.
  • Verify encoded and decoded bytes, bound output, and publish private caches atomically. Keep helper/DLL paths intact. Cache hits load no decoder.
  • Reuse the bounded-decoding approach from SWC #12741 and Aube #1930. Filesystem compression is not included.
  • Build, ESLint, and 13 compression tests passed in this rerun on macOS ARM64 / Node 26. The tests include real PTY loading, corruption, permissions, read-only packages, and concurrent publication.
  • Windows/x64 execution, Electron packaging/signing, and the full upstream CI matrix remain unverified locally. No npm publication performed.

Design and reproduction.

@jdalton

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

@jdalton

Copy link
Copy Markdown
Author

\cc Simon Lydell (@lydell)

@jdalton

John-David Dalton (jdalton) commented Nov 11, 2026 •

Copy link
Copy Markdown
Author

Okay, I updated the chart and confirmed that yes, this does not look worth it for this approach. #1016 shows more progress, however #1018 is where it will likely end up with Node 26 FFI usage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant